Host systems writing data to a persistent memory device or storage system, typically a persistent memory or non-volatile data memory media such as a magnetic hard disk drive (HDD), an electronic flash drive (EFD), or a solid state disk (SSD), may use what is known as a storage processor (SP) system to control how the input and output operations (IO) from the host system are transmitted to the storage system. The storage processors (SP) may have many parallel file systems operating in one, or in a plurality of SP systems, that perform the data mapping and data format translations needed for transmitting IO operations from the host system to the storage system. The SP file system operations may generate and also keep track of various pieces of information about the IO data that the SP file system is sending to the storage media. This data about the data being handled in the SP file system may be known as metadata. Metadata may be information about inode tables, block metadata (BMD), slice mapping, special block data, file type, and other forms of information about the data, and may be temporarily stored in a cache memory in the SP file system.
The SP file system may temporarily locally store the generated metadata to a cache buffer memory location in the SP file system, and the SP file system may mark the buffer as being dirty, since the metadata in the SP file system buffer is now not the same as in the storage system due to the handling by the SP file system, i.e., the data is not synchronized. The SP file system may perform what may be known as a sync operation, for example every 10 to 40 seconds to sync the metadata. These sync operations require that a selected portion of the total IO process bandwidth be dedicated to the IO needed for the sync operation, and thus may reduce the total number of IO processes available for host read and write (R/W) requests. This may increase the host latency time for IO operations in high usage situations.
The sync operation for the flushing of dirty metadata from a cache memory to the persistent memory needs to occur often enough for the cache to remain available, and yet not use so many IO operations that the host IO operations are delayed.